Comments for MEDB 5502, Week 06

Topics to be covered

What you will learn
- Test of two proportions
- Chi-square test of independence
- Odds ratio versus relative risk
- Concepts behind the logistic regression model
- Logistic regression with categorical variables
- Logistic regression with interactions

Comparing two binary outcomes

Is there a difference in the proportion of deaths between male passengers and female passengers on the Titanic?
Is there difference in the proportion of patients finishing the full three doses of HPV vaccine between Black women and White women?
Does using a ng tube for feeding in pre-term infants increase the probability of successful breast feeding at six months?

Other comparisons involving a binary outcome

Is there are difference in the proportion of deaths between first class, second class, and third class passengers?
Does age influence the proportion of women finishing the full three doses of HPV vaccine?
Controlling for the mother’s age, does using a ng tube for feeding in pre-term infants increase the probability of successful breast feeding at six months?

Hypothesis framework

\(H_0:\ \pi_1=\pi_2\)
\(H_1:\ \pi_1=\pi_2\)
Compute \(\hat p_1\) and \(\hat p_2\) from samples
Accept \(H_0\) if \(\hat p_1-\hat p_2\) is close to zero.
- \(T=(\hat p_1-\hat p_2)/s.e.\)
- 95% CI: \((\hat p_1-\hat p_2) \pm Z_{\alpha/2}s.e.\)

Data layout, 1 of 2

Data layout, 2 of 2

Confidence interval and test of hypothesis

Live demo, Test of two proportions

Break #1

What you have learned
- Test of two proportions
What’s coming next
- Chi-square test of independence

Chi-square test of independence, 1 of 2

Equivalent to test of two proportions
Lay out data in two by two table

\[\begin{matrix} & No\ event & Event \\ Treatment & O_{11} & O_{12}\\ Control & O_{21} & O_{22} \end{matrix}\]

Chi-square test of independence, 2 of 2

\[\begin{matrix} & No\ event & Event \\ Treatment & E_{11} = n_1 (1-\hat p_.) & E_{12}=n_1 \hat p_.\\ Control & E_{21} = n_2 (1-\hat p_.) & E_{22}=n_2 \hat p_. \end{matrix}\]

\(X^2=\Sigma \frac{(O_{ij}-E_{ij})^2}{E_{ij}}\)

Example: Titanic survival by sex

Moderate or large sample size: Pearson Chi-Square
Small sample size: Fisher’s Exact test

Live demo, Chi-square test of independence

Break #2

What you have learned
- Chi-square test of independence
What’s coming next
- Odds ratio versus relative risk

Titanic data

       Survived   Died  Total
Female   308      154     462
Male     142      709     851
Total    450      863   1,313

Titanic data, odds of death

       Survived   Died  Total  Odds
Female   308      154     462  2     to 1 against
Male     142      709     851  4.993 to 1 in favor
Total    450      863   1,313

Odds ratio = 4.993 / 0.5 = 9.986

Titanic data, probability of death

       Survived   Died  Total  Probability
Female   308      154     462    0.3333
Male     142      709     851    0.8331
Total    450      863   1,313

Relative risk = 0.8331 / 0.3333 = 2.5

Which is better

Relative risk is consistent with how most people think, but
- Relative risk cannot always be computed
- Relative risk has an ambiguity

Fractions are funny

  ----------  ----------
  0.8  (4/5)  1.25 (5/4)  
  0.75 (3/4)  1.33 (4/3)  
  0.67 (2/3)  1.50 (3/2)  
  0.50 (1/2)  2.00 (2/1)  
  ----------  ----------

Swapping the numerator and denominator

Odds ratio = male odds / female odds
- = 4.993 / 0.5 = 9.986
Odds ratio = female odds / male odds
- = 0.5 / 4.993 = 0.1001
Relative risk = male probability / female probability
- = 0.8331 / 0.3333 = 2.4996
Relative risk = female probability / male probability
- = 0.3333 / 0.8331 = 0.4001

Sometimes when we are comparing two groups, we’ll put the first group in the numerator and the other in the denominator. Sometimes we will reverse ourselves and put the second group in the numerator.

An odds ratio that you compute with males in the numerator and females in the denominator would be 9.986 or about 10. You could, however, just as easily put the females in the numerator and males in the denominator. Then your odds ratio would be 0.1001 or about 1/10. The two fractions are equivalent. Multiplying the odds by a factor of 10 going from females to males is the multiplying the odds by a factor of 1/10 going from males to females.

This ambiguity also appears for relative risks. The probability of death increases by 2.5 if you put males in the numerator, and the probability of death decreases by a factor of 0.4 if you put females in the numerator. Think of these as reciprocals: 2.5 is 5/2 and 0.4 is 2/5.

The numbers may look quite different but as long as you remember what the reciprocal fraction is, you shouldn’t get too confused.

Interpretability, 1 of 3

Change from 25% probability to 50% probability
Change from 3 to 1 odds against to even odds
- RR = 2, OR = 3

Interpretability, 2 of 3

Change from 25% probability to 75% probability
Change from 3 to 1 odds against to 3 to 1 odds in favor
- RR = 3, OR = 9

Interpretability, 3 of 3

Change from 10% probability to 90% probability
Change from 9 to 1 odds against to 9 to 1 odds in favor
- RR = 9, OR = 81

Designs that rule out the use of the relative risk, 1 of 2

         Cancer cases  Controls  Total  
Balding       72          82      154  
 Hairy        55          57      112  
 Total       129         139      268

Some research designs, particularly the case-control design, prevent you from computing a relative risk. A case-control design involves the selection of research subjects on the basis of the outcome measurement rather than on the basis of the exposure.

Consider a case-control study of prostate cancer risk and male pattern balding. The goal of this research was to examine whether men with certain hair patterns were at greater risk of prostate cancer. In that study, roughly equal numbers of prostate cancer patients and controls were selected. Among the cancer patients, 72 out of 129 had either vertex or frontal baldness compared to 82 out of 139 among the controls (see table below).

In this type of study, you can estimate the probability of balding for cancer patients, but you can’t calculate the probability of cancer for bald patients. The prevalence of prostate cancer was artificially inflated to almost 50% by the nature of the case-control design.

So you would need additional information or a different type of research design to estimate the relative risk of prostate cancer for patients with different types of male pattern balding.

Designs that rule out the use of the relative risk, 2 of 2

         Heart disease     Healthy     Total  
Balding    127 (9.4%)   1,224 (90.6%)  1,351  
 Hairy     548 (6.7%)   7,611 (93.3%)  8,159  
 Total     675          8,835          9,510

Contrast this with data from a cohort study of male physicians (Lotufo et al 2000). In this study of the association between male pattern baldness and coronary heart disease, the researchers could estimate relative risks, since 1,446 physicians had coronary heart disease events during the 11-year follow-up period.

For example, among the 8,159 doctors with hair, 548 (6.7%) developed coronary heart disease during the 11 years of the study. Among the 1,351 doctors with severe vertex balding, 127 (9.4%) developed coronary heart disease (see table below). The relative risk is 1.4 = 9.4% / 6.7%.

You can always calculate and interpret the odds ratio in a case control study. It has a reasonable interpretation as long as the outcome event is rare (Breslow and Day 1980, page 70). The interpretation of the odds ratio in a case-control design is, however, also dependent on how the controls were recruited (Pearce 1993).

Covariate adjustments

          Children   No children  Total  
Epilepsy  232 (40%)   354 (60%)    586  
Control    79 (72%)    30 (28%)    109  
Total     311         384          695

Another situation which calls for the use of odds ratio is covariate adjustment. It is easy to adjust an odds ratio for confounding variables; the adjustments for a relative risk are much trickier.

In a study on the likelihood of pregnancy among people with epilepsy (Schupf and Ottman 1994), 232 out of 586 males with idiopathic/cryptogenic epilepsy had fathered one or more children. In the control group, the respective counts were 79 out of 109 (see table below).

The simple relative risk is 0.55 and the simple odds ratio is 0.25. Clearly the probability of fathering a child is strongly dependent on a variety of demographic variables, especially age (the issue of marital status was dealt with by a separate analysis). The control group was 8.4 years older on average (43.5 years versus 35.1), showing the need to adjust for this variable. With a multivariate logistic regression model that included age, education, ethnicity and sibship size, the adjusted odds ratio for epilepsy status was 0.36. Although this ratio was closer to 1.0 than the crude odds ratio, it was still highly significant. A comparable adjusted relative risk would be more difficult to compute (although it can be done as in Lotufo et al 2000).

Ambiguous and confusing situations

One hundred pound sack of potatoes
- 99% water, 1% potato
- Weighs 1 pound after completely drying
- Instead dry until 2% potato
  - How much does it weigh then?

The relative risk can sometimes produce ambiguous and confusing situations. Part of this is due to the fact that relative measurements are often counter-intuitive. Consider an interesting case of relative comparison that comes from a puzzle on the Car Talk radio show. You have a hundred pound sack of potatoes. Let’s assume that these potatoes are 99% water. That means 99 parts water and 1 part potato. These are soggier potatoes than I am used to seeing, but it makes the problem more interesting.

If you dried out the potatoes completely, they would only weigh one pound. But let’s suppose you only wanted to dry out the potatoes partially, until they were 98% water. How much would they weigh then?

The counter-intuitive answer is 50 pounds. 98% water means 49 parts water and 1 part potato. An alternative way of thinking about the problem is that in order to double the concentration of potato (from 1% to 2%), you have to remove about half of the water.

Relative risks have the same sort of counter-intuitive behavior. A small relative change in the probability of a common event’s occurrence can be associated with a large relative change in the opposite probability (the probability of the event not occurring).

Example: physician recommendations

                 No cath      Cath      Total  
 Male patient   34  (9.4%)  326 (90.6%)  360  
Female patient  55 (15.3%)  305 (84.7%)  360  
         Total  89          631          720

Consider a recent study on physician recommendations for patients with chest pain (Schulman et al 1999). This study found that when doctors viewed videotape of hypothetical patients, race and sex influenced their recommendations. One of the findings was that doctors were more likely to recommend cardiac catheterization for men than for women. 326 out of 360 (90.6%) doctors viewing the videotape of male hypothetical patients recommended cardiac catheterization, while only 305 out of 360 (84.7%) of the doctors who viewed tapes of female hypothetical patients made this recommendation.

The odds ratio is either 0.57 or 1.74, depending on which group you place in the numerator. The authors reported the odds ratio in the original paper and concluded that physicians make different recommendations for male patients than for female patients.

A critique of this study (Schwarz et al 1999) noted among other things that the odds ratio overstated the effect, and that the relative risk was only 0.93 (reciprocal 1.07). In this study, however, it is not entirely clear that 0.93 is the appropriate risk ratio. Since 0.93 is so much closer to 1 and 0.57, the critics claimed that the odds ratio overstated the tendency for physicians to make different recommendations for male and female patients.

Although the relative change from 90.6% to 84.7% is modest, consider the opposite perspective. The rates for recommending a less aggressive intervention than catheterization was 15.3% for doctors viewing the female patients and 9.4% for doctors viewing the male patients, a relative risk of 1.63 (reciprocal 0.61).

This is the same thing that we just saw in the Car Talk puzzler: a small relative change in the water content implies a large relative change in the potato content. In the physician recommendation study, a small relative change in the probability of a recommendation in favor of catheterization corresponds to a large relative change in the probability of recommending against catheterization.

Thus, for every problem, there are two possible ways to compute relative risk. Sometimes, it is obvious which relative risk is appropriate. For the Titanic passengers, the appropriate risk is for death rather than survival.

Example: Breast feeding study

           Continued bf  Stopped bf  Total  
Treatment   19 (37.3%)   32 (62.7%)    51  
 Control     5  (8.8%)   52 (91.2%)    57  
  Total     24           84           108

But what about a breast feeding study. Are we trying to measure how much an intervention increases the probability of breast feeding success or are we trying to see how much the intervention decreases the probability of breast feeding failure? For example, Deeks 1998 expresses concern about an odds ratio calculation in a study aimed at increasing the duration of breast feeding. At three months, 32/51 (63%) of the mothers in the treatment group had stopped breast feeding compared to 52/57 (91%) in the control group.

While the relative risk of 0.69 (reciprocal 1.45) for this data is much less extreme than the odds ratio of 0.16 (reciprocal 6.2), one has to wonder why Deeks chose to compare probabilities of breast feeding failures rather than successes. The rate of successful breast feeding at three months was 4.2 times higher in the treatment group than the control group. This is still not as extreme as the odds ratio; the odds ratio for successful breast feeding is 6.25, which is simply the inverse of the odds ratio for breast feeding failure.

One advantage of the odds ratio is that it is not dependent on whether we focus on the event’s occurrence or its failure to occur. If the odds ratio for an event deviates substantially from 1.0, the odds ratio for the event’s failure to occur will also deviate substantially from 1.0, though in the opposite direction.

Break #3

What you have learned
- Odds ratio versus relative risk
What’s coming next
- Concepts behind the logistic regression model

What is logistic regression?

Binary outcome
Categorical or continuous predictors
Linear on the log odds scale

The logistic regression model is a model that uses a binary (two possible values) outcome variable. Examples of a binary variable are mortality (live/dead), and morbidity (healthy/diseased). Sometimes you might take a continuous outcome and convert it into a binary outcome. For example, you might be interested in the length of stay in the hospital for mothers during an unremarkable delivery. A binary outcome might compare mothers who were discharged within 48 hours versus mothers discharged more than 48 hours.

The covariates in a logistic regression model represent variables that might be associated with the outcome variable. Covariates can be either continuous or categorical variables.

For binary outcomes, you might find it helpful to code the variable using indicator variables. An indicator variable equals either zero or one. Use the value of one to represent the presence of a condition and zero to represent absence of that condition. As an example, let 1=diseased, 0=healthy.

Why log odds?

Statistical model of surgery
- Estimates probability of demise
- First prediction: probability=1.2
Log odds prevent out of range predictions

A logistic regression model examines the relationship between one or more independent variable and the log odds of your binary outcome variable. Log odds seem like a complex way to describe your data, but when you are dealing with probabilities, this approach leads to the simplest description of your data that is consistent with the rules of probability.

Let’s consider an artificial data example where we collect data on the gestational age of infants (GA), which is a continuous variable, and the probability that these infants will be breast feeding at discharge from the hospital (BF), which is a binary variable. We expect an increasing trend in the probability of BF as GA increases. Premature infants are usually sicker and they have to stay in the hospital longer. Both of these present obstacles to BF.

A linear model for probability, 1 of 2

A linear model of probability, 2 of 2

A multiplicative model for probability

It’s worthwhile to consider a different model here, a multiplicative model for probability, even though it suffers from the same problems as the additive model.

In a multiplicative model, you change the probabilities by multiplying rather than adding. Here’s a simple example.

Figure 3. Hypothetical probabilities from a multiplicative model

In this example, each extra week of GA produces a tripling in the probability of BF. Contrast this to the linear models shown above, where each extra week of GA adds 2% or 3% to the probability of BF.

A multiplicative model can’t produce any probabilities less than 0%, but it’s pretty easy to get a probability bigger than 100%. A multiplicative model for probability is actually quite attractive, as long as you have good reason to expect that all of the probabilities are small, say less than 20%.

The relationship between odds and probability

odds = prob / (1-prob)
prob = odds / (1+odds)
- \(0 \le\) prob \(\le 1\)
- \(0 \le\) odds \(\le \infty\)
  - \(0 \le\) odds against \(\le 1\)
  - \(1 \le\) odds in favor \(\le \infty\)

Another approach is to try to model the odds rather than the probability of BF. You see odds mentioned quite frequently in gambling contexts. If the odds are three to one in favor of your favorite football team, that means you would expect a win to occur about three times as often as a loss. If the odds are four to one against your team, you would expect a loss to occur about four times as often as a win.

You need to be careful with odds. Sometimes the odds represent the odds in favor of winning and sometimes they represent the odds against winning. Usually it is pretty clear from the context. When you are told that your odds of winning the lottery are a million to one, you know that this means that you would expect to having a losing ticket about a million times more often than you would expect to hit the jackpot.

It’s easy to convert odds into probabilities and vice versa. With odds of three to one in favor, you would expect to see roughly three wins and only one loss out of every four attempts. In other words, your probability for winning is 0.75.

If you expect the probability of winning to be 20%, you would expect to see roughly one win and four losses out of every five attempts. In other words, your odds are 4 to 1 against.

The formulas for conversion are

odds = prob / (1-prob)

and

prob = odds / (1+odds).

In medicine and epidemiology, when an event is less likely to happen and more likely not to happen, we represent the odds as a value less than one. So odds of four to one against an event would be represented by the fraction 1/5 or 0.2. When an event is more likely to happen than not, we represent the odds as a value greater than one. So odds of three to one in favor of an event would be represented simply as an odds of 3. With this convention, odds are bounded below by zero, but have no upper bound.

A log odds model for probability, 1 of 4

A log odds model for probability, 2 of 4

A log odds model for probability, 3 of 4

A log odds model for probability, 4 of 4

An example of a log odds model with real data, 1 of 3

An example of a log odds model with real data, 2 of 3

log odds = -16.72 + 0.577 \(\times\) GA

An example of a log odds model with real data, 3 of 3

log odds = -16.72 + 0.577 \(\times\) 30 = 0.59
odds = exp(log odds) = 1.8
prob = odds / (1+odds`) = 0.64

Let’s examine these calculations for GA = 30. The predicted log odds would be the intercept plus the slope times 30.

Convert from log odds to odds by exponentiating.

And finally, convert from odds back into probability.

prob = 1.80 / (1+1.80) = 0.643

The predicted probability of 64.3% is reasonably close to the true probability (77.8%).

You might also want to take note of the predicted odds. Notice that the ratio of any odds to the odds in the next row is 1.78. For example,

3.20/1.80 = 1.78

5.70/3.20 = 1.78

It’s not a coincidence that you get the same value when you exponentiate the slope term in the log odds equation.

exp(0.59) = 1.78

This is a general property of the logistic model. The slope term in a logistic regression model represents the log of the odds ratio. This represents the increase (decrease) in risk as the independent variable increases by one unit.

Live demo, Concepts behind the logistic regression model

Break #4

What you have learned
- Concepts behind the logistic regression model
What’s coming next
- Logistic regression with categorical variables

Categorical variables in a logistic regression model, 1 of 3

1st class odds: 129/193 = 0.67 or 193/129 = 1.5
2nd class odds: 161/119 = 1.35 or 119/161 = 0.74
3rd class odds: 573/138 = 4.15 or 138/573 = 0.24

Categorical variables in a logistic regression model, 2 of 3

1.50 / 0.24 = 6.212
0.74 / 0.24 = 3.069

Categorical variables in a logistic regression model, 3 of 3

0.74 / 1.50 = 0.494
0.24 / 1.50 = 0.161

If you prefer to do the analysis with each of the other classes being compared back to first class, then select FIRST for reference category.

This produces the following output:

Here the pclass(1) row provides an odds ratio of 0.494 which equals 0.74 / 1.50. The odds against dying are about half in second class versus first class. The pclass(2) provides an odds ratio of 0.161 (approximately 1/6) which equals 0.24 / 1.50. The odds against dying are 1/6 in third class compared to first class. The Constant row provides an odds of 1.496 to 1 against dying for first class.

Notice that the numbers in parentheses (pclass(1) and pclass(2)) do not necessarily correspond to first and second classes. It depends on how SPSS chooses the indicator variables. How did I know how to interpret the indicator variables and the odds ratios? I wouldn’t have known how to do this if I hadn’t computed a crosstabulation earlier. It is very important to do a few simple crosstabulations before you run a logistic regression model, because it helps you orient yourself to the data.

Live demo, Logistic regression with categorical variables

Break #5

What you have learned
- Logistic regression with categorical variables
What’s coming next
- Logistic regression with interactions

Interactions in logistic regression

Odds ratios vary by a third factor
Interpretation is more tedious

Odds ratios for first class

Odds ratio for second class

Odds ratio for third class

Logistic regression with interaction

Odds ratio for 3rd class = 4.608
Odds ratio for 1st class = 4.608 \(\times\) 6.572 = 30.2
Odds ratio for 2nd class = 4.608 \(\times\) 9.289 = 42.8

Line plot for interaction, 1 of 2

Line plot for interaction, 2 of 2

Live demo, Logistic regression with interactions

Summary

What you have learned
- Test of two proportions
- Chi-square test of independence
- Odds ratio versus relative risk
- Concepts behind the logistic regression model
- Logistic regression with categorical variables
- Logistic regression with interactions

Additional topics??

Learning objectives
- Define Logistic Regression analysis
- Compare and Contrast Multiple Regression analysis and Logistic Regression analysis
- List and define the three types of logistic Regression analysis
- Define how predictor variables are selected
- Define Multicollinearity
- List and explain the problems that Multicollinearity causes if present in a Logistic Regression analysis
- List and explain the methods for selecting predictor variables
- List and explain the problems that too many predictors can cause in a Logistic Regression analysis
- Compare and contrast Logistic Regression analysis with Linear and Multiple Regression analysis
- Explain the model equation for a Logistic Regression analysis
- Explain the problem with the Assumption of Linearity for a Logistic Regression analysis
- Explain the method that is used to overcome the problem with the Assumption of Linearity
- Define Maximum Likelihood Estimation
- Explain Probability
- Explain Odds
- Explain the conversion of probability to odds
- Define and explain the model equation for a Logistic Regression analysis
- Define reference standard
- Identify the value that represents the fit of the model
- Define Log likelihood
- List and define the 4 different calculations for R squared for an Logistic Regression analysis
- Define and explain the Wald Statistic
- Explain the importance of Odds Ratios in a Logistic Regression analysis
- Identify Exp(B)
- Identify and define the methods for predictor variable input in a Logistic Regression analysis
- Understand which predictor variable input method is best used
- Identify, define and explain the testing for all assumptions
- Identify and explain all “Important Considerations” for a Logistic Regression analysis
- Calculate Sensitivity, Specificity, PPV and NPV using the information found in the SPSS Classification Table
- Explain the meaning of Block 0
- Explain the meaning of all subsequent blocks (following Block 0) in a Logistic Regression analysis
- Interpret an Odds Ratio for a continuous predictor variable
- Interpret an Odds Ratio for a categorical predictor variable
- Calculate the sensitivity and specificity of the model
- Explain what an ROC identifies in a Logistic Regression analysis
- Create a complete write up for a Logistic Regression analysis
- Know the table that must be included in a write up for a Logistic Regression analysis